Thermal to Visible Face Recognition

In low light conditions, visible light face identification is infeasible due to the lack of illumination. For nighttime surveillance, thermal imaging is commonly used because of the intrinsic emissivity of thermal radiation from the human body. However, matching thermal images of faces acquired at nighttime to the predominantly visible light face imagery in existing government databases and watch lists is a challenging task. The difficulty arises from the significant difference between the face’s thermal signature and its visible signature (i.e. the modality gap). To match the thermal face to the visible face acquired by the two different modalities, we applied face recognition algorithms that reduce the modality gap in each step of face identification, from low-level analysis to machine learning techniques. Specifically, partial least squares-discriminant analysis (PLS-DA) based approaches were used to correlate the thermal face signatures to the visible face signatures, yielding a thermal-to-visible face identification rate of 49.9%. While this work makes progress for thermal-to-visible face recognition, more efforts need to be devoted to solving this difficult task. Successful development of a thermal-to-visible face recognition system would significantly enhance the Nation’s nighttime surveillance capabilities.